Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix multi discrete #4338

Merged
merged 4 commits into from
Mar 13, 2019
Merged

Fix multi discrete #4338

merged 4 commits into from
Mar 13, 2019

Conversation

stefanpantic
Copy link
Contributor

@stefanpantic stefanpantic commented Mar 12, 2019 •
edited
Loading

What do these changes do?

  • Fixes a bug with calculating log_rhos in VTrace.
  • Arguments for the function get_log_rhos were switched and instead of calculating target_action_log_probs - behaviour_action_log_probs, the reverse was being done, behaviour_action_log_probs - target_action_log_probs. I swithed the arguments and also replaced the elementwise list comprehension with an equivalent tensor operation.
  • Reverts Revert "[wingman -> rllib] IMPALA MultiDiscrete changes (#3967)" #4332

Related issue number

@stefanpantic stefanpantic mentioned this pull request Mar 12, 2019
Merged
@bjg2 bjg2 mentioned this pull request Mar 12, 2019
Closed
@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12795/
Test FAILed.

@ericl
Copy link
Contributor

ericl commented Mar 13, 2019

Looks good.

  • IMPALA_BreakoutNoFrameskip-v4_0_env=BreakoutNoFrameskip-v4: RUNNING, [33 CPUs, 1 GPUs], [pid=35223], 1029 s, 82 iter, 5238000 ts, 326 rew

This is exactly in line with the reference result on rl-experiments repo.

@ericl
Copy link
Contributor

ericl commented Mar 13, 2019

Pushed a lint fix.

@ericl ericl self-assigned this Mar 13, 2019
@ericl ericl merged commit 2202a81 into ray-project:master Mar 13, 2019
@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12830/
Test FAILed.

@nikola-j nikola-j deleted the fix-multi-discrete branch March 27, 2019 09:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ericl ericl ericl approved these changes

Assignees

@ericl ericl

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@stefanpantic @AmplabJenkins @ericl